Hybrid HMM/MLP models for times series prediction

A hard problem in time series analysis is often the non-stationarity of the series in the real world. However an important sub-class of nonstationarity is piecewise stationarity, where the series switch between different regimes with finite number of regimes. A motivation to use this model is that each regime can be represented by a state in a finite set and each state match one expert i.e. a multilayer perceptron (MLP). Addressing this problem, we present a class of models consisting of a mixture of experts, so we have to find which expert does the best prediction for the time series. For example A.S. Weigend et al. [6] introduce a gating network to split the input space. However in this study, we use instead a hidden Markov chain, because it is a powerfull instrument to find a good segmentation, and is therefore usefull in speech recognition. The potential advantage of hidden Markov chains over gating networks is that the segmentation is only local with gating networks (it decides the probability of a state only with it’s inputs), but is global with a hidden Markov chain (the probability of the states at each moment depends on all the observations). So we will use this model for the time series forecasting, which has never been done when the model functions are non-linear functions represented by different MLPs.